Incremental search algorithms for on-line planning

FACULTY OF ENGINEERING, SCIENCE AND MATHEMATICS SCHOOL OF ELECTRONICS AND COMPUTER SCIENCE Doctor of Philosophy by Jason D. R. Farquhar An on-line planning problem is one where an agent must optimise some objective criterion by making a sequence of action selection decisions, where the time and resources used in making decisions count in assessing overall solution quality. Typically in these problems it is not possible to find an optimal complete solution before an initial action must be executed, instead to maximise its performance the agent must interleave decision making and execution. This thesis investigates using decision theoretic techniques to solve these problems by equipping the agent with the ability to reason about the “complexity induced” uncertainties in its information and the costs of computation. The basic thinking/execution interleaving is provided by incremental search, where decisions are made incrementally based upon a guided partial search through the local space of possible solutions. The major sub-problems such an agent must solve are; i) decision making—how to make decisions whilst in a state of “complexity induced uncertainty”, ii) search control—which node to expand next, iii) stopping—when to stop searching. Decision making is treated as a value estimation problem. By representing the agent’s uncertainty in probabilistic terms this can be solved using decision theoretic techniques. Existing decision making systems are analysed and new low computational cost approximate decision theoretic algorithms developed. These are shown to give significant improvements in decision quality. Search control is also treated as an estimation problem, in this case estimating the expected value of computation (EVC), which is the expected benefit of a computation in reducing the agent’s uncertainty (hence improving action selection). New, low computational cost, approximations for the EVC are also developed and shown to give significant improvements in decision quality. Sophisticated stopping can be achieved by trading-off the EVC of further search against its computational cost. Experimental results show that integration of the sub-problem solutions is critical to agent performance. The results also show that (assuming no adverse interactions) improving decision making gives the greatest improvements, with search control and stopping offering more modest benefits.